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1.0  SUMMARY 


In  this  project,  we  designed  and  validated  a  new  concept  for  an  Oculometric  Motion 
Processing  Capability  Assessment  Tool  (OMCAT)  suitable  for  quickly  and  reliably  assessing  the 
visual  motion  processing  capabilities  of  airmen.  We  also  demonstrated  that  the  human 
performance  characteristics  captured  by  OMCAT  are  not  captured  by  acuity  measures.  The 
delivered  OMCAT  framework  is  ready  for  development  and  integration  into  a  prototype  system 
to  enable  the  assessment  of  the  OMCAT  suite  of  oculometric  measures  as  quantitative  predictors 
of  pilot  perfonnance  in  simulator  tests  in  support  of  Air  Force  vision  screening  applications. 

2.0  BACKGROUND  AND  OVERVIEW 

2.1  Hypotheses 

•  Hypothesis  #1: 

a.  A  small  set  of  oculometric  measures  can  be  found  to  succinctly  characterize  the 
visual  motion  processing  capabilities  of  individual  pilots,  of  their  cohort,  and  of 
an  aged-matched  normal  population,  and 

b.  These  oculometric  measures  can  be  used  to  detect  visual  motion  processing 
perfonnance  either  above  or  below  nonnal. 

•  Hypothesis  #2:  Some  of  our  oculometric  measures  will  conelate  with  operational 
perfonnance  in  OBVA  tasks.  Note  that  this  hypothesis  will  be  tested  in  the  OBVA 
laboratory  and  is  not  part  of  the  deliverable  for  this  project. 

2.2  Specific  Aims 

•  Tailor  NASA  Ames’s  current  laboratory  tasks  into  a  single  task  to  yield  a  10-minute 
“clinical”  test 

•  Develop  analyses  of  smooth  tracking  (pursuit)  eye  movements  to  measure  speed  and 
direction  discrimination 

•  Validate  a  small  set  of  perfonnance  measures  to  provide  repeatable  and  independent 
perfonnance  metrics 

•  Provide  a  database  of  baseline  human  performance 

•  Establish  engineering  specifications  for  a  future  OMCAT  system  from  our  hardware 
prototyping  efforts 
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3.0  METHODS  AND  SYSTEM  ENGINEERING  SPECIFICATIONS 


3.1  Rashbass  Arrows-In  Task 

We  used  a  two-dimensional  variant  of  the  Rashbass  step-ramp  paradigm  [1],  Subjects 
began  each  trial  by  fixating  the  tracking  target  (a  small  red  spot)  in  primary  position  and  pressing 
a  mouse  button.  Subjects  then  fixated  the  red  spot  for  a  randomized  duration  drawn  from  an 
exponential  distribution  [2]  (mean:  700  ms,  minimum:  200  ms,  maximum:  5000  ms)  to  defeat 
possible  temporal  expectation  of  motion  onset.  After  the  randomized  duration  had  elapsed,  the 
tracking  spot  made  a  small  step  [3]  in  a  random  direction  drawn  from  a  uniform  distribution 
around  the  unit  circle  (in  2-degree  increments)  and  immediately  began  moving  back  toward  the 
initial  fixation  location.  The  step  size  was  set  such  that  the  target  crossed  its  original  fixation 
location  200  ms  after  motion  onset.  On  each  trial,  the  velocity  of  the  target  spot  was  drawn  from 
a  distribution  of  five  speeds  (16,  18,  20,  22,  24  deg/s)  and  lasted  1.2  seconds.  Each  experiment 
consisted  of  180  tracking  trials  (~  1 5  minutes)  to  maintain  the  subject’s  alertness  [4]  and  allow 
for  a  clinically  reasonable  behavioral  session.  Stimuli  were  displayed  on  a  60-Hz  CRT  with 
resolution  of  1024  by  768  pixels  at  a  viewing  distance  of  57  cm,  which  subtended  an  area  of  38 
degrees  horizontal  by  29  degrees  vertical. 

3.2  Eye-Movement  Recording 

We  sampled  eye  position  at  240  Hz  with  an  ISCAN  eye-tracker  (ISCAN  Inc.,  Burlington, 
MA).  The  eye -tracker  camera  uses  a  256-  by  256-pixel  image  and  returns  the  centroid  of  the 
pupil  rounded  to  the  nearest  pixel.  The  eye-position  traces  were  calibrated  with  six  parameters 
(gain,  offset,  and  cross-terms  for  horizontal  and  vertical  [5])  fit  to  the  raw  digital  values  for 
fixations  at  nine  screen  locations,  which  yielded  a  precision  of  better  than  0.3°  (standard 
deviation  of  eye-position  while  fixating).  Saccades  were  detected  by  taking  the  correlation 
between  a  saccade-shaped  velocity  template  and  the  horizontal  eye  velocity  trace  [6],  which  we 
then  excised  from  the  velocity  traces  during  subsequent  analyses. 

3.3  Pursuit  Analyses 

We  used  a  “hinge”  model  [7]  to  mark  the  onset  of  the  pursuit  movement.  Because  our 
tracking  target  was  a  single  stimulus  moving  in  a  random  direction,  we  marked  pursuit  onset 
using  velocity  parallel  to  target  motion.  The  hinge  consists  of  two  line  segments  (baseline  and 
response,  each  100  ms  in  duration)  occurring  consecutively;  pursuit  latency  was  defined  as  the 
point  at  which  the  two  line  segments  intersected.  We  added  three  constraints  to  the  algorithm  to 
increase  its  robustness.  We  defined  the  baseline  velocity  to  be  zero,  constrained  the  acceleration 
to  be  positive,  and  modified  the  error  function  by  a  “recinonnal”  prior  probability  distribution. 
For  each  trial,  our  algorithm  measured  the  error  between  the  best  fit  of  the  two  free  parameters 
describing  the  hinge  model  (latency  and  acceleration)  to  the  velocity  trace  over  the  interval  from 
100  to  400  ms.  This  error  function  was  then  weighted  by  a  recinonnal  [8]  function  (mean:  5.4  s' 

,  standard  deviation  2  s'1)  with  its  median  at  the  expected  pursuit  latency  of  185  ms.  Pursuit 
latency  was  defined  to  be  the  time  that  yielded  the  minimum  error;  open-loop  acceleration  was 
defined  to  be  the  slope  of  the  response  interval.  Steady-state  gain  was  defined  to  be  the  ratio  of 
radial  eye  velocity  to  target  velocity  over  the  interval  from  400  to  700  ms  after  motion  onset,  and 
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the  proportion  of  smooth  pursuit  was  defined  to  be  the  ratio  of  eye  displacement  during  smooth 
pursuit  to  total  eye  displacement  over  this  interval.  We  also  measured  the  radial  direction  of  the 
pursuit  response  during  the  steady-state  interval  to  quantify  the  signal  and  noise  properties  of  the 
pursuit  response  to  angular  motion  as  described  in  detail  previously  [1]. 

4.0  RESULTS  AND  BASELINE  PERFORMANCE  DATA 

4.1  Defining  the  OMCAT  Oculometric  Measures 

We  established  four  different  categories  of  oculometric  performance  parameters. 

1 .  Initiation  (INIT)  measures:  After  detecting  and  removing  saccadic  intrusions  from  the 
eye -velocity  traces  [6],  we  measured  the  initial  latency  (INIT  latency)  of  the  pursuit 
response  (in  ms)  with  a  best- fitting  hinge  function  [7].  We  also  quantified  the  initial 
mean  acceleration  (INIT  ace)  in  the  first  100  ms  of  the  smooth  eye-movement  response 
(the  so-called  open-loop  interval)  [9].  These  provide  measures  of  the  vigor  of  the  initial 
pursuit  visual-tracking  response. 

2.  Steady-state  (SS)  measures:  All  measures  of  steady-state  tracking  were  made  in  the 
interval  from  400  to  700  ms  following  motion  onset.  We  measured  the  ratio  of  the 
steady-state  eye-speed  to  target-speed,  or  the  traditional  steady-state  gain  (SS  gain)  [3], 
the  mean  size  of  the  catch-up  saccades  (SS  sacc  amp),  and  the  proportion  of  smooth 
tracking  (SS  prop  smooth). 

3.  Direction- tuning  (DIR)  measures:  We  also  measured  the  directional  tuning  of  the  pursuit 
response  [1].  This  provided  a  measure  of  the  oblique  effect  (DIR  anisotropy),  horizontal- 
vertical  asymmetry  (DIR  asymmetry),  and  directional  uncertainty  (DIR  noise). 

4.  Speed-tuning  (SPD)  measures:  We  also  developed  a  new  method  for  simultaneously 
measuring  speed  tuning  during  the  same  task.  This  provided  two  new  oculometric 
measures:  speed  responsiveness  (SPD  slope)  and  speed  uncertainty  (SPD  noise). 

The  pursuit  initiation  and  steady-state  tracking  measures  characterize  the  trial-by-trial 
tracking  response  dynamics,  whereas  the  direction  and  speed-tuning  measures  characterize  the 
signal  and  noise  components  of  the  response  to  changes  in  target  direction  and  speed.  Two  of 
the  three  direction-tuning  measures  are  unique  in  that  they  capture  nonlinear  distortions  as 
opposed  to  properties  of  a  quasi-linear  response. 
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4.2  Summary  of  Individual  Data  and  Across-Trial  Variability 


Figure  1  shows  all  the  oculometric  data  for  a  single  observer  from  a  single  test  session. 
The  distributions  show  that  the  measures  are  well  behaved  (pseudo-Gaussian).  The  direction 
tuning  shows  a  well-behaved  sensitivity  to  target  direction  along  with  the  typical  oblique  effect 
documented  previously  [1],  The  speed  tuning  documents  a  well-behaved  sensitivity  to  target 
speed.  Figure  1  is  formatted  as  a  summary  output  data  figure  for  a  15-minute  clinical  test. 
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#  observers  #  observers 


4.3  Baseline  Population  Oculometrics  and  Across-Subject  Variability 


4.3.1  Initiation  Metrics  (Figure  2) 


Initiation  (INIT) 


0  50  100  150  200 

acceleration  (deg/s2) 


Figure  2.  Distribution  of  Initiation  Metrics.  The  median  latency  of  181  ms 
and  the  median  acceleration  of  122.5  deg/s2  are  indicated  by  the 
solid  vertical  red  line.  Dashed  lines  indicate  the  quartiles. 
Note  the  well-behaved  (pseudo-Gaussian)  nature  of  the 
distributions . 
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#  observers  #  observers  #  observers 


4.3.2  Steady-State  Metrics  (Figure  3) 
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4.3.3  Direction-Tuning  Metrics  (Figure  4) 
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4.3.4  Speed-Tuning  Metrics  (Figure  5) 
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4.4  Reliability  and  Across-Time  Variability 

The  inter-subject  variance  is  a  factor  of  approximately  four  greater  than  the  test/retest 
variance,  on  average,  across  six  candidate  metrics  (Figure  6).  A  non-parametric  analysis  of 
variance  (Kruskal- Wallis  test)  yielded  highly  significant  effects  of  subject  for  all  the  above 
metrics  at  better  than  the  p  <  0.0001  level,  confirming  the  reliability  of  these  metrics  as  measures 
of  inter-subject  variability  and  thus  their  usefulness  as  performance  metrics,  even  when  only  a 
single  test  session  is  possible.  This  is  a  crucial  requirement  for  any  usable  clinical  test. 
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4.5  Correlation 


This  correlation  matrix  in  Figure  7  shows  a  number  of  critical  findings.  First,  none  of  our 
oculometric  measures  is  correlated  with  either  acuity  or  age.  Thus,  they  provide  novel 
infonnation  about  the  visual  motion  processing  capabilities  of  our  subjects,  rather  than  visual 
motion  processing  impairments  due  to  either  acuity  or  age.  Second,  the  two  direction-tuning 
“nonlinear”  metrics  (DIRasymmetry,  DIRanisotropy)  are  in  a  class  by  themselves.  They  are  not 
correlated  with  any  of  the  other  measures  and,  oddly,  only  weakly  correlated  with  each  other. 
Thus,  these  two  DIR  measures  provide  independent  infonnation  about  the  subjects  not  captured 
by  the  other  “linear”  measures.  Third,  there  is  a  class  of  linear  metrics  that  is  loosely  correlated 
with  each  other,  although  there  are  some  interesting  exceptions  (e.g.,  latency  is  largely 
uncorrelated  with  gain).  Thus,  our  10  oculometric  measures  provide  at  least  2  and  perhaps  as 
many  as  4  largely  independent  measures  of  motion  processing  performance. 


INITaccel 
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INITIatency 
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SPDslope 
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Figure  7.  Correlation  Statistics  for  All  Pairs  of  Our  10  Oculometric 
Measures  Across  All  40  Subjects  Tested.  Grey  scale  plots 
the  r2  value  of  the  correlation  with  white  being  1 
and  black  0. 
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5.0  CONCLUSIONS 


We  have  successfully  completed  all  facets  of  the  OMCAT  specific  aims: 

•  We  have  developed,  refined,  and  validated  a  methodology  to  collect  and  compute  10 
oculometric  measures  of  human  speed  and  direction  perception  that  could  potentially  be 
used  as  predictors  of  piloting  perfonnance. 

•  We  have  demonstrated  that  reliable  measures  of  this  set  of  metrics  can  be  obtained  from  a 
single  15 -minute  test  session  using  a  novel  task  and  efficient  data  collection. 

•  We  have  demonstrated  that  all  10  of  these  measures  provide  information  that  is 
uncorrelated  with  (i.e.,  independent  of)  both  traditional  acuity  measures  and  age. 

•  We  have  demonstrated  that  these  measures  provide  multiple  independent  dimensions  of 
human  motion  perception  perfonnance  (at  least  two  and  perhaps  as  many  as  four 
independent  classes  of  oculometrics,  each  potential  predictors  of  pilot  perfonnance). 

•  We  have  established  a  database  of  the  nonnal  distribution  of  these  new  perfonnance 
measures  among  a  healthy  group  of  subjects  who  would  meet  existing  aircrew  vision 
standards. 
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LIST  OF  ABBREVIATIONS  AND  ACRONYMS 


DIR 

INIT 

NASA 

OBVA 

OMCAT 

SPD 

ss 


direct-tuning  measures 
initiation  measures 

National  Aeronautics  and  Space  Administration 
Operational  Based  Vision  Assessment 
Oculometric  Motion  Processing  Capability  Assessment  Tool 
speed-tuning  measures 
steady-state  measures 


12 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited.  Case  Number:  88ABW-2014-1794,  18  Apr  2014 


